Papers with deep neural model
The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding (2020.acl-demos)
Copied to clipboard
Xiaodong Liu, Yu Wang, Jianshu Ji, Hao Cheng, Xueyun Zhu, Emmanuel Awa, Pengcheng He, Weizhu Chen, Hoifung Poon, Guihong Cao, Jianfeng Gao
| Challenge: | MT-DNN is an open-source natural language understanding toolkit . it allows researchers and developers to train customized deep learning models . |
| Approach: | They present MT-DNN, an open-source natural language understanding toolkit . it is designed to facilitate rapid customization for a broad spectrum of NLU tasks . MT supports multi-task knowledge distillation, which can substantially compress a deep neural model without significant performance drop. |
| Outcome: | The proposed model can significantly compress a large model without significant performance drop. |
Visualizing the Relationship Between Encoded Linguistic Information and Task Performance (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that encoding more syntactic information does not lead to better performance. |
| Approach: | They propose a method to optimize pareto-optimal models by formalizing it as a multi-objective optimization problem. |
| Outcome: | The proposed method is better than a baseline method on two NLP tasks. |
Abstraction not Memory: BERT and the English Article System (2022.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained models are the state of the art in linguistics. |
| Approach: | They compare the performance of pre-trained and native English language models on the task of article prediction set up as a three way choice (a/an, the, zero) they argue that BERT captures a high level generalisation of article use akin to human intuition. |
| Outcome: | The proposed model outperforms humans on the linguistically interesting task of article prediction. |
Challenging Reading Comprehension on Daily Conversation: Passage Completion on Multiparty Dialog (N18-1)
Copied to clipboard
| Challenge: | Existing approaches to reading comprehension on multiparty dialogs have focused on children's stories or newswire. |
| Approach: | They propose a new corpus and a robust deep learning architecture for a task in reading comprehension on multiparty dialog. |
| Outcome: | The proposed model outperforms the state-of-the-art model on a different genre using bidirectional LSTM, showing a 13.0+% improvement for longer dialogs. |
A Richer-but-Smarter Shortest Dependency Path with Attentive Augmentation for Relation Extraction (N19-1)
Copied to clipboard
| Challenge: | Existing approaches to extract relationship between entities in sentences suffer from missing or redundant information. |
| Approach: | They propose a deep neural model that combines the advantages of the two approaches to extract the relationship between two entities in a sentence. |
| Outcome: | The proposed model outperforms baseline models on the SemEval-2010 dataset. |
Deep Exhaustive Model for Nested Named Entity Recognition (D18-1)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a task of finding entities with specific semantic types such as Protein, Cell, and RNA in text. |
| Approach: | They propose a deep neural model for nested named entity recognition . they enumerate all possible regions or spans as potential entity mentions . |
| Outcome: | The proposed model outperforms state-of-the-art models on nested and flat NER . it achieves 77.1% and 78.4% respectively in terms of F-score, without external knowledge resources. |
Common Ground Tracking in Multimodal Dialogue (2024.lrec-main)
Copied to clipboard
Ibrahim Khalil Khebour, Kenneth Lai, Mariah Bradford, Yifan Zhu, Richard A. Brutti, Christopher Tam, Jingxuan Tu, Benjamin A. Ibarra, Nathaniel Blanchard, Nikhil Krishnaswamy, James Pustejovsky
| Challenge: | In dialogue modeling, there is considerable attention on “dialogue state tracking” (DST) but “common ground tracking” identifies the shared belief space held by all participants in a task-oriented dialogue: the task-relevant propositions all participants accept as true. |
| Approach: | They propose a method for automatically identifying the current set of shared beliefs and ”questions under discussion” of a group with a shared goal. |
| Outcome: | The proposed method predicts moves toward building common ground relative to ground truth in a multimodal interaction with an AI. |
Improving Fine-grained Entity Typing with Entity Linking (D19-1)
Copied to clipboard
| Challenge: | Existing methods for fine-grained entity typing require a large tag set and knowledge of the context. |
| Approach: | They propose a deep neural model that uses context and information from entity linking to improve fine-grained entity typing. |
| Outcome: | The proposed model achieves 5% absolute strict accuracy improvement over the state of the art on two datasets. |
Textual Backdoor Attacks Can Be More Harmful via Two Simple Tricks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing textual backdoor attacks are vulnerable to backdoors . researchers add extra training task to distinguish poisoned and clean data . |
| Approach: | They propose two tricks that make existing backdoor attacks much more harmful . first trick is to add an extra task to distinguish poisoned and clean data . second trick is using all the clean training data rather than the original clean data. |
| Outcome: | The proposed tricks can significantly improve attack performance in three tough situations including clean data fine-tuning, low-poisoning-rate, and label-consistent attacks. |